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MULTIMEDIA SURVEILLANCE AND MONITORING SYSTEM 
INCLUDING INTERNET CONFIGURATION 

Inventor: 
David A. Monroe 

BACKGROUND OF THE INVENTION 
pmh of the Invention. The subject invention is directed to surveillance and monitoring 
systems and is specifically directed to a comprehensive, hybrid multimedia surveillance system 
based on wireless data Emission and video streaming techniques, still image and/or step video, 
video streaming, audio, motion detection, event detection and/or physical condition detection using 
wireless communications and Internet communication techniques and methods. 

TW„ gc inn of the Prior Art. Video monitoring and surveillance of locations or areas for 
security, safety monitoring, process control, and other such applications by use of closed circuit 
television and similar systems has been in widespread use for many years. The cost of these 
systems has come down significantly in recent years as the camera and monitor components have 
steady dropped in cost while increasing in quality. As a result, these systems have proliferated 
in their application and are proving extremely useful for both commercial and residential 
applications. 

These "closed circuit television" systems typically consist of a monochrome or color 
television camera, a coaxial cable, and a corresponding monochrome or color video monitor and 
power sources for the cameras and monitors. The interconnection of the camera and monitor is 
typically accomplished by the use of coaxial cable, which is capable of carrying the 2 to 10 
megahertz bandwidths of closed circuit television systems. Further, the coaxial cable is shielded 
for noise immunity, is relatively inexpensive and is commonly available. There are several 
limitations to coaxial cable supported systems. First, the signal is attenuated by the cable in 



proportion to the distance traveled. Long distance video transmission on coaxial cable requires 
expensive transmission techniques. Second, both the cable, per se, and the installation is 
expensive. Both of these limitations limit the use of coaxial closed circuit systems to installations 
requiring less than a few thousand feet of cable. Third, when the cable cannot be concealed is not 
only unsightly, but is also subject to tampering and vandalism. 

Other hardwired systems have been used, such as fiber optic cable and the like, but have 
not been widely accepted primarily due to the higher costs associated with such systems over 
coaxial cable. Coaxial cable, with all of its limitations, remains the system of choice to the present 
day. Also available are techniques using less expensive and common twisted pair cable such as 
that commonly used for distribution of audio signals such as in telephone or office intercom 
applications. This cable is often referred to as UTP (unconditioned twisted pair) or SUTP 
(shielded unconditioned twisted pair) cable. Both analog and digital configurations are available. 
W Both analog and digital techniques have been implemented. This general style of twisted pan- 
cable is : also widely used in Local Area Networks, or LAN's, such as the lOBase-T Ethernet 
system, 100 Base-T and later systems. Newer types of twisted pair cable have bee developed that 
have lower capacitance and more consistent impedance than the early telephone wire. These 
newer types of cable, such as "Category 5" wire, are better suited for higher bandwidth signal 
transmission and are acceptable for closed circuit video applications. By way of example, typical 
audio signal are approximately 3 kilohertz in bandwidth, whereas typical video signals are 3 
megahertz in bandwidth or more. Even with the increased bandwidth capability of this twisted 
pair cable, the video signals can typically be distributed directly over twisted pair cable only a few 
hundred feet. In order to distribute video over greater distances, video modems 
© (modulator/demodulators) are inserted between the camera and the twisted pair wiring and again 
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between the twisted pair wiring and the monitor. Twisted pair cable is lower in cost than coaxial 
cable, is easer to install than coaxial cable and, like coaxial cable, is readily available. 

Wireless systems utilizing RF energy are also available. Such systems usually consist of 
a low power UHF transmitter and antenna system compatible with standard television monitors 
or receivers tuned to unused UHF channels. The FCC allows use of this type of system without 
a license for very low power levels in the range of tens of milliwatts. This type of system 
provides an economical link but does not provide transmission over significant distances due to 
the power constraints placed on the system. The advantage of this system over hardwired systems 
is primarily the ease of installation. However, the cost is usually much higher per unit, the 
number of channels is limited and system performance can be greatly affected by building 
geometry or nearby electrical interference. Further, the video is not as secure as hardwired 
systems. The may be picked up by anyone having access to the channel while in range of the 
~ transmitter and is thus, easily detected and/or jammed. 

Because of the inherent limitations in the various closed circuit television systems now 
available, other media has been employed to perform security monitoring over wider areas. This 
is done with the use of CODECs (coder/decoders) used to reduce the bandwidth. Examples include 
standard voice bandwidth telephone circuits, more sophisticated digital telephonic circuits such 
as frame relay or ISDN circuits and the like. While commonly available and relatively low in 
cost, each of these systems is of narrow bandwidth and incapable of carrying "raw" video data 
such as that produced by a full motion video camera, using rudimentary compression schemes to 
reduce the amount of data transmitted. As previously discussed, full motion video is typically 3 
to 7 megahertz in bandwidth while typical low cost voice of data circuits are 3 kilohertz in 
bandwidth. 
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There are known techniques for facilitating full motion video over lower bandwidth 
circuits. The video teleconferencing (VTC) standards currently in use are: Narrow Band VTC 
(H.320); Low Bitrate (H.324); ISO-Ethernet (H.322); Ehternet VTC (H.323): ATM VTC 
. (H.321); High Resolution ATM VTC (H.3.10). Each of these standards has certain advantages and 
disadvantages depending upon the volume of data, required resolution and costs targets for the 
system. These are commonly used for video teleconferencing and are being performed at typical 
rates of 128K, 256K, 384K or 1.544M bit for industriaycommercial use. Internet teleconferencing 
traditionally is at much lower rates and at a correspondingly lower quality. Internet VTC may 
be accomplished at 33.3KBPS for example. Video teleconferencing is based on video 
compression, such as the techniques set forth by CCITT/ISO standards, Internet standards, 
Proprietary standards or by MPEG current standards. A discussion of these standards may be 
found at the web site 

»/^;//c«^ ° ther ' S ° metimeS 

proprietary, schemes using wavelet or motion JPEG compression techniques and the like are also 
in existence. There are a number of video teleconferencing and video telephone products available 
for trar^mitting "full motion" (near real-time) video over these circuits such as, by way of 
example, systems available from AT&T and Panasonic. While such devices are useful for their 
intended purpose, they typically are limited in the amount of data which may be accumulated 
and/or transmitted because they do not rely on or have limited compression, and, therefore, are 
time consuming and require large capacity in order to manage a small amount of information, 
particularly if such information is to be archived. There are also devices that transmit "live" or 
in near real-trme over the Internet, such as QuickCam2 from Connectix, and more recently, CU- 
0 See-Me and Intel products utilizing the parallel printer port, USB port, ISA, PCI card, or 
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PCMCIA card on a laptop computer. Many of these are public access systems and have neither 
the resolution required or the security required to provide for good surveillance systems. 
NetMeeting from Microsoft and Proshare software packages form Intel also provide low quality 

personal image distribution over the Internet. 

All of the current Internet low cost products have the ability to transmit motion or "live- 
video. However, such products are limited or difficult, if not impossible, to use for security 
applications because the resolution of the compressed motion video is necessarily low because of 
limited original resolution of the sample and the applications of significant levels of video 
compression to allow use of the low bandwidth circuits. The low resolution of these images will 
not allow positive identification of persons at any suitable distance form the camera for example. 
The low resolution would not allow the reading of an automobile tag in another example. 

In many security applications it is desirable to monitor an area or a situation from a 
monitor located many miles from the area to be surveyed. As stated, none of the prior art systems 
readilFaccommodates this. Wide band common carriers such as are used in the broadcast of high 
quality television signals could be used, but the cost of these long distance microwave, fiber or 

satellite circuits is prohibitive. 

SUMMARY OF THE INVENTION 
The subject invention is directed to a comprehensive, wireless multimedia surveillance and 
monitoring system which is adapted for transmitting event data, video and/or image monitoring 
information, audio signals and other sensor and detector data over significant distances using 
digital data transmission over a wireless LAN (WLAN), Intranet or Internet for automatic 
assessment and response including dispatch of response personnel. Both wired and wireless sensor 
0 systems may be employed. GPS dispatching is used to locate and alert personnel as well as to 
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indicate the location of one or more events. Automatic mapping, dispatch and response vectoring 
permits rapid response. The wireless LAN connectivity permits local distribution of audio, video 
and image data with relatively high bandwidth without requirement of a license and without 
relying on a common carrier and the fees associated therewith. The surveillance system may be 
interfaced with a WAN (wide area network) or the Internet for providing a worldwide, low cost 
surveillance system with virtually unlimited geographic application. Centralized monitoring 
stations have access to all of the surveillance data from various remote locations via the Internet 
or the WAN. A server provides a centralized location for data collection, alarm detection and 
processing, access control, auto response generation, paging, automatic e-mail generation, 
telephone dialing and message transmission, dispatch processing, logging functions, configuration 
management, and/or other specialized functions. The server may be inserted virtually anywhere 
in the mtranet/lnternet network. The topology of the network will be established by the 
geographic situation of the specific installation. Appropriate firewalls may be set up as desired to 
protect unauthorized access to the system or collected data. The server based system permits a 
security provider to have access to the sensor and surveillance data or to configure or reconfigure 
the system from any station on the intranet or Internet. 

The system of the subject invention permits monitoring of locations over great distances 
with sufficient resolution to provide widespread use as a security surveillance device. Suitable 
configurations of basic electronics for supporting this system are shown and described in my co- 
pending applications, as follows: MICRORIT I, MICRORIT II, FAXCAM RJ-lland GROUND 
BASED SECURITY AND WIRELESS AIRCRAFT SECURITY. All of these applications are 
fully incorporated herein by reference. The subject invention is specifically directed to a system 
that can transmit essential information for surveying and monitoring a selected zone or area. The 
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system includes video and/or image sensors as well as audio, condition and/or event monitoring 
systems. In its preferred form, the comprehensive multi-media safety and surveillance system of 
the subject invention provides both visual and audio information as well as critical data such as 
fire and smoke detection. Manually operated transducers, such as panic buttons, door contacts, 
floor sensors, and the like may also be included to activate the system in the presence of an event 
at the sensor location, such as a fire alarm or security alarm or the like. In my aforementioned 
copending applications, incorporated herein by reference, detection and sensor systems are utilized 
to provide monitoring stations or personnel, such as security personnel, and/or a base monitoring 
station critical information from the sensor system and to record the information and permit 
reconstruction of events after the fact. The system of the subject invention permits detection of 
unexpected events, breach of security, and other activities in the vicinity of any sensor within the 
system and identifies the time and location of the event for permitting an appropriate response. 
A GPS system may be included to provide accurate positioning information of both the sensors 
and roving or mobile response units such as security personnel. Steerable video cameras may be 
incorporated in order to monitor movements in the range of the sensors. The cameras may be 
activated and directed based on the location data provided by the integral GPS system. It is also 
desirable to include focusing and timing functions so that selective sequencing, zoom and axial 
(x,y,z) positioning can be utilized. 

In its preferred form, a plurality of sensor units, which may include at least one video 
image sensor/device and/or at least one audio sensor and/or at least one motion sensor, are placed 
strategically about the facility to be monitored. In addition, strategically placed motion detectors, 
fire sensors, panic switches, smoke sensors and other monitoring equipment are incorporated in 
the system. Cameras may be placed throughout the facility and in other desired spaces including 
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on the ground outside the facility. The audio sensors/transducers and other sensors and detectors 
are also strategically located both internal and external of the facility. 

While the system may be hardwired, in its preferred for the system of the present invention 
is adapted for use in connection with wireless transmission and receiving systems. The wireless 
system is particularly useful for adapting the system as a retrofit in existing facilities and also 
provides assurances against disruption of data transmission. In the preferred embodiment, the ; 
wireless system is fully self-contained with each sensor unit having an independent power supply 
and, where required for image sensors, a sensor light source. The security system may include 
either motion sensitive, audio sensitive and/or image processing based activation systems so that 
the equipment is not activated until some event is detected, i.e., the system is action triggered. 

In the preferred embodiment, the system will transmit any detected information to a 
monitor system located at a base monitoring station, located on site and/or at a remote location, 
and/or a server for logging, forwarding, archiving same. The base station has instant live access 
to all of the image and audio signals as they are captured by the sensors, and where desired is 
adapted to record and make an historic record of the images for archive purposes. Where random 
access recording techniques are used, such as, by way of example, digital random access memory 
storage devices, the information by be readily searched for stored information. 

If unauthorized personnel breaches the security area and the audio and video equipment is 
activated, signals will be immediately transmitted to the base station, usually with an alert signal 
to attract the attention of base personnel. This will give immediate access to information 
identifying the activity, the location and the personnel involved. Further, in the preferred 
embodiment of the invention, an appropriate response system will be activated for securing the 
immediate area and taking counter measures to protect the security of the area. This may include 
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dispatch of personnel, sealing off the area, and/or, where appropriate, transmitting an audible 
and/or visual alarm as well as instructions. 

In the preferred embodiment, when a large number of sensors are utilized in a complex 
system, the plurality of sensors may be synchronized through an integral multiplexing system 
whereby the plurality of data, including visual image data, may be displayed, recorded, and/or 
transmitted in either a split screen or serial fashion. A time or chronology signal may also be 
incorporated in the data scheme, whereby all collected real time streaming media on individual 
events are time stamped for exact time and date. Any signal which is capable of being captured 
and stored may be monitored in this manner. 

Utilizing the wireless system of the invention in combination with the battery back-up 
power supply, it is possible to continue collecting information without using a central or public 
power source. This assures that the system will operate even if power is disrupted for any reason 
such as, by way of example, tampering by unauthorized personnel. The sensors can detect power 
outages**! generat alarm conditions as reported over the LAN or WLAN. In its simplest form, 
only triggered sensors are active, and only the signals generated thereby are transmitted to the 
security station. In the preferred embodiment, a combination of hardwired and wireless 
devices and components will be used. One advantage to use of certain wireless components is that 
the capture, retrieval, monitor and archive system utilizing a wireless trammitting/receiving 
system assures that transmission will not be lost in wires in a portion of the system are cut or 
otherwise interrupted, during a fire or an earthquake, for example. Wireless configurations are 
also particularly desirable for retrofit installations where it may be difficult to install cable. 
Further, in addition to ease of installation, wireless components are virtually portable and can be 
re-deployed based on history of need within a given installation simply by moving the component 
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to a new location. In the preferred embodiment, components of such a system would be completely 
self-contained with an integrated power supply and, as required for image sensors, an integrated 
illumination system. The Mumination system would provide lighting to permit capture of images 

in the event the public power system fails. 

Of course, it is an important aspect of the invention that all of the collected data, including 
any video images, be recorded to provide an historic video record. This will prove invaluable as 
an aid in reconstructing the events in a "post mortem" investigation. 

The system of the present invention is capable of transmitting the collected information 
over significant distances using typical voice bandwidth carriers in sufficient resolution to 
accommodate security surveillance and other high resolution applications. In one 
embodiment, one or more wireless cameras or sensor devices are in communication with a local 
Intranet system using a wireless LAN (local area network) connection. A monitoring station is 
also in communication with the Intranet at any desired location on the LAN. The monitoring 
station can monitor audio and/or video and/or image data and/or sensor data continuously, 
periodically as programmed, or upon event detection such as by motion detection, contact closure 
or detection by an independent system that is in communication with the surveillance system. The 
. wireless LAN connectivity permits local distribution of audio, video and/or image data over a 
relatively high bandwidth without requirement of a license and without relying on a common 
carrier and the fees associated therewith. 

Where longer distance transmission is required, the surveillance system of the subject 
invention may be interfaced with a WAN (wide area network) or the Internet. This provides a 
worldwide, low cost surveillance system with virtually unlimited geographic application. Such 
a system is very useful in applications where multiple buildings are part of the surveillance 
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network, such as, by way of example, a college campus, school builings or distircts, or corporate 
campus or a government installation. Centralized monitoring stations can then have access to all 
of the surveillance data from various remote locations via the Internet or the WAN. 

In an enhancement of the invention, a security server is added to the system for expanding 
and enhancing the capability and functionality of the surveillance system. The server provides 
a centralized location for data collection, alarm detection and/or processing, access control, 
dispatch processing, logging functions, data mining capability, configuration and management 
functions, map serving, and other specialized functions. The sever may be inserted virtually 
anywhere in the Intranet/Internet network as it may be accessed universally. The topology of the 
network will be established by the geographic situation of the installation. Multiple servers may 
be employed. The server permits the implementation of standard Internet tools and techniques 
such as TCP/IP , HTML and browser support that will allow nearly universal access to the system 
with proper security access codes. Appropriate access controls and firewalls may be set up as 
desired. The server based system permits a security provider to have access to the sensor and 
surveillance data and/or to configure or reconfigure the system from any station on the Internet, 
such as from a PC at home. The system supports and manages the collection, logging and 
archiving of data; data mining; the monitoring, assessment, response and dispatching of alarm 
conditions; data distribution to remote locations; routing; data format conversion as necessary; and 
signals the dispatch of response support where required. 

An example of a multiple location, server based, Internet support system in accordance 
with the subject invention is a typical school district. Using the subject invention, wireless sensors 
may be located in strategic areas in each school building or on each campus of a district. A 
wireless receiver (or multiple receivers) is then connected to the local area network on the campus 
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and a monitor station is placed at a strategic location, for example, in the administration office. 
The collected data may be displayed on a PC monitor or other monitor such as a CRT console 
monitor or an LCD, personal communications devices , and/or any of a variety of suitable monitors 
and display units. In addition, the data is sent via the Intranet or Internet to a server located, for 
example, at the district office, where all campuses are monitored. The server can also distribute 
and dispatch information upon the occurrence of an event. For example, if a panic button were 
to be activated at a sensor station, this signal would be immediately transmitted, assessed and a 
response initiated. If the panic button indicated a fire, a fire response team would be dispatched 
directly to the scene, as well as other appropriate responsive actions. In a worst case example, 
if a gun shot were detected, using acoustic the server would be able to identify the sensor on the 
specific campus where the event occurred and then send the information to appropriate authorities 
such as, by way of example, on site roving guards, the closest police station and the closest fire 
station, providing a fast response after the occurrence of the event. The wireless nature of the 
sensors also imnimizes the likelihood of tampering with the signals. The multiple communication 
paths provide redundancy in the system making it unlikely that all monitoring stations would be 
down at one time. The system is a cost-effective, flexible and comprehensive system for 
enhancing the campus and building security issues currently facing most institutions and 
organizations. 

One significant advantage to the system of the subject invention is that it accommodates 
multimedia surveillance in applications and locations where physical wiring cannot be used and 
over distances not possible with other systems. The system of the present invention provides 
surveillance capability utilizing techniques ranging from closed-circuit, hard wired systems to the 
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Internet and is not dependent upon the limitation of data capability ; distance attenuation and costs 
associated with systems currently on the market. 

It is an important feature of the invention that it is adapted to the use of non-localized 
wireless carrier links. WLAN systems are generally restricted to distances ranging from 200 to 
4000 feet. Greater distances can use common carriers for transmitting the data. CDPD data 
service, Internet two-way pager service or an Internet satellite service and the like can be used. 
Even carriers generally not including an Internet gateway can be used if modified to provide an 
access path to the server or to the Internet gateway. For example, conventional (non-digital) 
cellular service can be provided with an ISP connection to provide a gateway to the Internet. This 
would permit a remote unit at a location such as at a distant soccer field or the like to be in contact 
while on rounds. Type of event, location and multimedia data can be dispatched to the mobile 
unit for initiating immediate action. 

The server is the heart of the surveillance system an monitors the status of the wired and/or 
wireless sensors as well as the status of the monitor stations anywhere on the system. The sever 
monitors event detection and both manages and monitors response dispatch. The server also 
manages the collection, dissemination, logging and/or archiving of data. The server manages, 
monitors, configures and reconfigures the system components. This can be done seamlessly from 
a remote location, eliminating the requirement that personnel attend to each station except when 

hardware upgrades are required. 

The server supported system permits a broad range of signal and event processing. For 
example, the server can arm and disarm sensors for detection of external events, audio detection 
or video detection based on predetermined schedules or manual control via Internet or Intranet 
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access. If an activated sensor is triggered, the server can dispatch alarm conditions in several 
ways, including by not limited to sending: 

messages to specifically assigned monitoring stations or mobile units; 
a telephone call to a designated land line (such as 911) with an audio message 

describing the event, 
a telephone call to a designated wireless telephone number with an audio message 

describing the event in order to dispatch a mobile unit; 

a numeric pager message with transmitted number signifying a voice mailbox; 

a voice pager with audio message; 

a text page describing the event; 

a text message to wireless personal data assistant (PDA); 

e-mail to specific addresses; 

graphic information showing, for example, a map of the event location to PDA . 
The server based system can be used to notify multiple parties, or can notify and wait for 
a confirmation and then perform programmed sequenced steps based on response of the notified 
stations. Complex alert "decision tree" sequences can be implemented for personnel notification 

in any desirable hierarchy. 

The multimedia surveillance system of the subject invention permits high resolution still 
image transmission as well as full motion monitoring and, where higher resolution is required, 
step video. All three types of data may be delivered in combination to maximize the quality of the 

data collected. 

It is, therefore, an object and feature of the subject invention to provide a wireless 
communication link between sensors and/or monitors. 
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It is another object and feature of the subject invention to provide a multimedia 
surveillance system for transmitting video and image data over significant distances using typical 
voice bandwidth carriers such as the public telephone system, and wireless carriers such as cellular 
telephones, including AMPS, PCS, GSM and the like, CDPD data links, two-way pagers, Iridium 
satellites and the like. 

It is also an object and feature of the subject invention to provide a multimedia surveillance 
system adapted for utilizing wireless video and/or image data collection and/or transmission using 

the Internet and/or IP protocols. 

It is also an object and feature of the subject invention to utilize Intranet and Internet 
communication systems to distribute surveillance data and control data. 

It is another object and feature of the subject invention to provide a security surveillance 
system adapted for use in connection with a wireless LAN (WLAN) communications system. 

It is also an object and feature of the subject invention to provide a multimedia surveillance 
system-adapted for making a permanent record of collected data in desired sequence and format 
such.as before, during and after event detection, through programmed monitoring, event response 
and human control and intervention. 

It is a further object and feature of the invention to provide mapping and other graphic 
information based on event detection and location data. 

It is yet another object and feature of the invention to provide vectoring capability for 
guiding response personnel to specific locations in response to events and for guiding non- 
responding or at risk personnel away from such events. 

It is a further object and feature of the invention to provide remote management and 
configuration capability of a multimedia surveillance system. 
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It is a further object and feature of the invention to provide a server supported multimedia 
surveillance system having an Intranet and Internet compatible server for data retention, alarm 
processing, configuration management, access control, access logging, "crosspoint switching" of 
data, motion detection, scene analysis, scheduled activation and deactivation detection, display 
data distribution and sequencing and general control and management 

It is a further object and feature of this invention to provide a comprehensive, multi-media 
surveillance and security system for monitoring one or more selected zones form a remote 
location. 

It is also an object and feature of the subject invention to provide communications between 
the monitored zone and a surveillance station using wireless communication techniques. 

It is another object and feature of the subject invention to provide a comprehensive, multi- 
media data generating, collecting, displaying, transmitting, receiving and storage safety and 
surveillance scheme for security and surveillance. 

It is an additional object and feature of the subject invention to provide a video and/or 
audio and/or data record of events occurring for archival and retrieval purposes. 

It is yet another object and feature of the subject invention to provide apparatus for 
permitting security personnel to receive video images, audio information and data relating to 

critical components and areas. 

It is an additional object and feature of the subject invention to provide interspersed full 

motion and still video for image surveillance. 

It is also an object and feature of the invention to provide location information of both the 
personnel and the event in order to dispatch appropriate response personnel in closest proximity 
of the event. 
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Other objects and features will be readily apparent from the accompanying drawings and 
detailed description of the preferred embodiments. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figs. 1-7, 14a-f, and A are directed to various system configurations, as follows: 
Fig. 1 is a diagrammatic illustration of a wireless system connected to an Intranet. 
Fig. 2 is a diagrammatic illustration of the system of Fig. 1 further connected to a remote 

station via the Internet. 

Fig. 3 illustrates a wireless LAN configuration and includes a system server. 

Fig. 4 illustrates a server based system using Internet communications. 

Fig. 5 illustrates various wireless carrier communication link configurations with gateway. 

Fig. 6 is an expansion of Fig. 5 and includes wireless carrier links to monitors via wireless 

carrier and the Internet. 

Fig. 7 is a comprehensive multimedia surveillance system incorporating the features 
illustrated in Figs. 5 and 6 with multiple, potentially different, wireless carriers. 

Figs.Wa, 14b and 14c are illustrations of various system configurations for LAN systems. 

Figs. 14d, 14e and 14f are illustrations of various system configurations for. wireless local 

area network (W-LAN) systems. 

Fig. A is an illustration of a system configuration combining wired and wireless 
components with wired and wireless gateways to a common carrier and the Internet. 

[Fig. 15 is to be deleted]. 

Figs. B, C, 18, D-P, 9, 10, 11, 12, 13, 16a-c, and 17 are directed to sensor and 
component (appliance) configurations, as follows: 

Fig. B is an illustration of a hardwired IP fire alarm multimedia station. 
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Fig. C is a wireless configuration of the unit shown in Fig. B. 

Fig. 18 is a perspective view of a multimedia alarm and response module for use in 
connection with the subject invention. 

Fig. D is an illustration of a basic IP video camera. 

Fig. E is a block circuit diagram for the IP video camera of Fig. D. 

Fig. F is an illustration of a signaling transducer unit. 

Fig. G is a block circuit diagram for the signaling transducer unit of Fig. F. 

Fig. H is an illustration of a sensing/signaling unit. 

Fig. I is a block circuit diagram for the sensing/signaling unit of Fig. H. 

Fig. J is an expanded block diagram for a highly integrated sensing/signaling unit. 

Fig. K is an illustration of a hardwired IP video intercom. 

Fig. L is a block circuit diagram for the hardwired IP video intercom of Fig. K T 

Fig. M is an illustration of a wireless IP video intercom. 

Fig. N is a block circuit diagram for the wireless IP video intercom of Fig. M. 

Fig. O is a block circuit diagram of a contact/LAN interface with two channels 

Fig. P is a block circuit diagram of an analog sensor/LAN interface. 

Fig. 9 is a block diagram of a camera sensor for use in connection with the multimedia 
surveillance system to the display of the subject invention. 

Fig. 10 is a flow chart showing the flow of data from a camera through the processing 
system of a multimedia surveillance system to the display in accordance with the subject 

Fig. 11 is similar to Fig. 10 with the addition of a control processor. 

Fig. 12 shows and alternative configuration using a high performance DSP with the 
embedded functions of Figs 10 and 12. 
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Fig. 13 is a diagrammatic illustration of a collateral triggering device for use in connection 
with the multimedia surveillance system of the subject invention. 

Fig. 16 is an integrated sensor/wireless LAN subsystem using DSP technology. 

Fig. 17 is a perspective view of a multimedia camera tracking system for use in connection 

with the subject invention. 

Figs . through are directed to streamed video and still imaging technology 

as incorporated in the subject invention, as follows: 



Figs . through are directed to server functions including: (a) system 

configuration schemes, (b) clusters, (c) dispatch systems, (d) data logging and (e) data 
mining, as follows: 

Fig. Q is an illustration of a clustered alarm workgroup scheme. 
Fig. R is an illustration of a hybrid conventional logging system with LAN system 

combination. 

Figs. S through are directed to conventional system interfaces, as follows: 

Fig. S is an illustration of a telephone system gateway configuration. 
Fig. 8 is an illustration of the server dispatch capabilities for a system configured with the 
capabilities shown in Fig. 7 including telephone access, voice response and voice mail. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
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A basic wireless surveillance system is shown in Fig. 1. In this configuration a wireless camera 
or video sensor 20 is located at one remote location and a second wireless camera or video sensor 
22 is located at a second remote location. The sensor 20 is shown as a video camera for purposes 
of brevity. It is an important feature of the invention that each sensor may include any 
combination of a variety of sensors or detectors for providing audio, visual, condition and event 
detection at the sensor operational zone. A more detailed description of specific sensor and 
detector capability is shown and described in my copending applications entitled: (WIRELESS 
AIRCRAFT AND GROUND BASED SURVEILLANCE APPLICATIONS). 

In a typical operation sequence, the camera/sensors are polled, sending a high resolution image 
periodically. These images are archived on one or more servers. If an event happens, a full 
motion stream may be activated from one or more of the camera/sensors in the area of the event. 
The vide stream is routed to one or more monitor stations, one or more roving PDA monitor 
stations and is "recorded" on one or more servers in digital format. Sequential camera operation 
is accomplished by directing a series of identified cameras to swithch on and off their respective 
video streams in a predefined or programmed sequence. This emulates the timed sequential video 
switches commonly used for security surveillance. The sequence can be easily configured and 
reconfigured at the server. For example, a map of the sensors can be displayed. The sensors can 
be activated in the order they will be viewed. Switching can be by name, number or map location, 
using point-and-click technology. Once and automatic sequence is enabled, manual activation may 
be accocomplished by signaling a camera to turn OFF of deactivate its video stream and another 
to turn ON. This can also be accomplished by keyboard or other input command such as by a 
mouse click on the map locator. 
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The data is compressed based on need. Still images may be specified with low-loss 
compression such as wavelet, to preserve as much information as possible. Motion video may be 
specified with higher-loss compression to reduce the bandwidth requirements MPEG-1 or motion 
JPEG, for example could be specified for motion sequences. When the system is supporting a 
"non-alarm" condition of operation, the available network bandwidth may be allocated to support 
a maximum number of streaming feeds at lower resolutions. If a critical alarm condition occurs, 
not might a streaming feed be initiated from a sensor, an existing feed may be commanded to 
increase its data bandwidth for higher resolution. This may require reducing the bandwidth and 
resolution of other less critical feeds, or switching them into an OFF mode. Thus a completely 
dynamic bandwidth control scheme may be utilized to properly balance the number of streaming 
sensors, and the resolution and bandwidth of those sensors against the surveillance monitoring 
requirements at the scene of an event. 

Positioning of the cameras may also have an effect on what resolution and bandwidth is 
needed. For example, a camera in a small space such as an elevator does not need to have as 
much resolution as a camera in a large area such as an auditorium or a cafeteria. The bandwidth 
of the cameras can, therefore, be allocated based on need, with the elevator camera programmed 
for less bandwidth/resolution and the cafeteria camera with more bandwidth/resolution. 

Diagrams of various system configurations for LAN and wireless local area network (W- 
LAN) systems are shown in Figs. 14a, 14band 14c. As shown in Fig. 14a, with a camera sensor 
CI for purposes of simplification, the camera CI generates an analog signal which is converted 
to a digital signal at converter 400 and then compressed at the motion video compressor 402. This 
can be accomplished by industry standard techniques such as motion-JPEG, MPEG, or motion 
wavelet compression or other current or future compression algorithms. The compressed digital 
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signal is then packetized by the LAN interface 404 and transmitted to the LAN 206 in well-known 
manner. An analog audio sensor such as microphone 419 is added in Fig. 14b and is supported 
the dedicated converter 406 and compressor 408 for input to the multiplexer 410 where the 
compressed digital audio signal is combined with the compressed digital video signal to produce 
a complex multi-media signal for packetization by the LAN 404 interface. As shown in Fig. 14c, 
digital sensors such as motion detector 431 may also be included. The motion detector digital 
signal does not require conversion and is input directly into the multiplexer 410. As also shown 
in Fig. 14c, the LAN may be wireless, with a wireless transceiver 412 being incorporated in the 
system. As previously described, any portion of the system may be wired or wireless depending 
on ease of installation, mobility requirements and other issues. It may be noted that functions such 
as the motion video compressor, audio compressor, multiplexer and LAN protocol functions may 
all be performed as software and could operate on one high speed computer such as a Digital 

Signal Processor (DSP). 

Turning now to Fig. 15, additional multi-media sensors may be incorporated in the 
system, as well, and may be wireless or hard wired as appropriate. For example, one or more 
audio sensors such as voice sensor 413 transmit audio signals to multiplexer processor 232. 
Various function sensors, such as, by way of example, an entire array of intrusion security sensors 
415 may also be incorporated in the multi-media system of the subject invention. Where a plurality 
of such sensors are utilized, it is desirable to provide a local multiplexer system 238 to mimmize 
the amount of duplicative hardware. In the example shown, all of the intrusive security sensors 
in array 415 require only a single transmitter and antenna as part of a local multiplexer 238 which 
may then feed a combined signal to the multimedia multiplexer 232. In a wireless system, the 
Q security sensor array may also be fully self-contained with an independent power supply. 
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As shown, a variety of image sensor devices may be incorporated, including the video 
cameras CI, C2, C3...Cn, an advanced imaging device such as an infrared camera 220 and the 
like. All of these produce a visual signal. In addition, various audio signals may be incorporated 
utilizing a variety of audio sensor devices, such as a voice sensor 413, two-way radios 224, 226 
and a public address system 228. All of these produce an audio signal. The operational data signals 
are also incorporated, as previously described, and may include the GPS sensor 472, other 
navigational sensors 230, the various intrusion sensors 415 and other sensors 425. Thus, the 
system of the subject invention will accommodate a multiple input, multi-media array 
incorporating video, audio and digital data signals into a comprehensive database for providing 
detailed information relating to the condition of the monitored area at any time. 

Each sensor device signal is introduced into a multi-media multiplexer network 232 which 
includes a image multiplexer subsystem 234, a dedicated audio multiplexer subsystem 236 and a 
digital data multiplexer subsystem 238, all of which produce distinctive multiplexed signals which 
are introduced into a master multiplexer subsystem 232 for producing a combined, comprehensive 
output signal, as selected, on each of lines 231, 233 and 235. It may also perform decompression 
functions for compressed command streams and compressed audio or video. The setup and 
control of the comprehensive output signal is provided by a master controller 241 and input to the 
multiplexer 232 at 243. The system controller receives commands and streaming audio information 
from other system elements and distributes them to controlled devices. The controller performs 
a command decoding function to sort out command and data streams directed toward specific 
devices and components of the system. 

The visual and textual data is available at a display monitor 454. The audio signal is 
output at 237 to an audio output system such as amplified speaker 240. All of the data, including 
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all video, audio and digital data will be recorded on the recorder system 470. Information 
representing audio, video, sensor data, and other vital digital data is fed from the multimedia 
multiplexer to the recorder 470 over the signal lines 233. It should be noted that the multimedia 
multiplexer may be analog, digital, or packetized digital data type, or a combination of 
technologies based on application. Where desired, selected portions of the systems data may be 
linked to the base station as the combined, comprehensive output signal on line 246 to be 
transmitted via an RF transmission system or the like as indicate by the radio system 480 and the 
antenna 482. As previously described, the information may also be transmitted to a wireless 
satellite via transceiver 280 and dedicated antenna 282. Once the information is generated as a 
useable data signal, as indicated at line 231, 233 and 235, the controller, in combination with 
commands from monitor stations or servers, controls the collection, monitoring and review of the 
information. This permits access to any single sensor signal, or any combination via line 231 by 
sending a command via line 248 to the controller 241 for controlling the monitor related 
multiplexing switches via line 243 to control the signal output on line 231. For example, this may 
be a single camera view or an array of intrusive motion sensors 115. 

Where desired, a light level detector may be is used for detecting light conditions such as 
the ambient lighting or transient conditions such as vehicle headlights or a flashlight. The light 
detector analog/digital converter adapts the ambient light levels into a digital data stream. This 
digitizer runs at near real-time rates for "real-time" illumination monitoring. The light detector 
signal processor can be programmed to look for profiles such as rapidly increasing or decreasing 
light conditions that may indicate a vehicle or a flashlight as opposed to the rising or setting sun. 
Detection of such and event would trigger a specified unique alarm condition to be transmitted 
back to other elements of the system. 
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External contact sensors may also be deployed and a condition change may be detected and 
processed by the contact signal processor. These may be devices such as door contacts, special 
motion detectors such as trip wires and the like, floor pads and the like which can be connected, 
either by wires or wireless means to the contact detection circuit. Detection of such an event 
would trigger a specified alarm condition to be transmitted back to other elements of the system. 

An audible speaker system can also be provided in the preferred embodiment and can 
provide numerous audio outputs such as, by way of example, voice output or a siren. This is a 
multi-function device and can be activated by local detection events, and by other system elements 
such as detection by a companion sensor unit signaling over the wireless system. The siren can 
indicate an area of concern, serve as a signal to security personnel and/or scare of intruders. The 
audible speaker can also be used to provide voice instructions or signals base on local detection 
events, and by other system elements. The controller produces the synthesized or stored voice 
signals. The controller can programmed or downloaded over the wireless system. The speaker 
system can also be use as a paging system by sending digitized or compressed voice signals over 
the wireless system to one or more multi-media devices. In addition, the audio speaker can be use 
conjunction with the audio detector 408 to communicate with the area. 

Power is provided in the well-known manner. In the preferred embodiment, system power 

is used to power up the system through a convenor and a rechargeable battery system comprising 
a charger/controller and rechargeable battery supply. 

In certain applications it may be desirable to combine many of the functions described 
herein, such as the signal processing, data multiplexing 232, LAN or WAN network transceiver 
280, control and parts of the network interface, perhaps utilizing software running at high speed 

in a high speed DSP engine. This would serve to reduce hardware complexity, improve 
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rebability, reduce power —on, and reduce cost, The network interface provides a wued 
interface to the system for connecting other system eiements in a hardwired configuration. Ms 
ean be any one of severai well known out evolving technologies such as lOBase-T, the better 100 
Base-T or high-speed Gigabit LAN or WAN ««, Such a configuration does no, depart 
from the scope and spirit of the subject invention. 

Fig. 16 is a diagrammatic illustration of an mtegrated sensor/wireless LAN subsysKm 
usta g DSP technoiogy. As there shown, the various analog sensors such the Ugh. sensor 300, me 
„re sensor 302, the humidity sensor 304, and dte sound or audio sensor 306 (as well as 
other sensors as previously described herein and as desired for appUcation) produce anatog signais 
which are converted at one of me dedicated anaiog-to-digital convenors 3!0 and men introduced 
into a mu.t*le*er 312. The multiplexer 3!2 produces a combined digital output signal which is 
Educed into the DSP processor 314, which produces the system output on Une 3X5, where it 
is again converted a, convenor 3!6, ampiified a. amplifier 318 and transmitted ™ antenna 320. 
^•preferred embodiment, an mfcgral power supply 322 is provided. Tne Sensor I/D address 
is on One 324. This system provides a highly mtegrated sensor/processor/transceiver and « 
can be housed on a single chip using available configuration technology. 
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^ sensors and/or audio sense* synchronized by a masfcr ******* so-, each sensor 
recorders. Omer sensors m a y a.so be incorporate hr the system, -h as motion sense, smoke and 

^er minimize the possibility of a ma— a. the onset eta cabbie occurrence. 

Fig nisaperspecUveviewofapreferredembodimentof tracking camera sensor 5 10. In 
ffi e preferred embodiment, the cameras are adapted to respond, several different types of centre,- 

signals, including but not limited to: 

X-axis position control as indicated by X-axis servomotor 550; 
X-axis position control as indicated by Y-axis servomotor 554; 
Lens zoom control as indicated by motorized zoom lens 558; and 

Mscontrolasindicatedbyiriscontrolle, (Theirismay also be automated). 

. k a„ w is mounted on a tilt mount 554 (y-axis) and 
mounting the system at location. The system body 552 vs mounted 

• / ji™*imrt md tiltine (V direction) of the camera for 
pan mount 550 (x-axis), permitting panmng (x drrecton) and tutrag ly 

— . wide are, A motorized zoom iens 55S is provided <* direction, The preferred 
embodiment me system also includes an audio sensor such as directional microphone 560. me 

fo m the surrounding are. The collected audio can be processed to detect potential emergency 
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^on.apsas.o^fo.^.P— of a perpetrator. The calcmated ongm ^ 

positions, and mose assets may be automatically dispatched. The audio anafog/digital convex 
^ the acoustic sign, representing me audio en— into a digita, data stream. The 

*.« for real-time audio monitoring. The audio signal 
digitizer runs at real-tane rates for realtime 

processor/compressor has two functions, ft is programme to perform detection in a numher of 
■W for that conmuontohetransmittedbac. to other elements of the system. Other typesof 

staati on or condition. For audio surveiUance applications, me compressor can also be used to 
provi de bandwidm reduction for audio transmission. !n mis application, the amount of data 
.presenungareal-timeaudiosheamwouldbereducedby using audio compression techmcmes arch 

i -.u^n tv»; c allnw^ better bandwidth utilization 
as LPC-10, or other well-known or proprietary algonthms. Tms allows better 

of the wireless and wired communications channels used by the system. 

mum mation means such as me infrared illuminator 562 permits surveillance during low 
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An ^grated OPS receiver 564 is provided for generating location information. TH, is 
particularly useful for -drop-and-place" sensors as opposed to permanent sensors. Other features 
su chasalaserr ang e fi „der566u n atcan ffi easure disfcnce to objects/personnel may be incorporated 
,„ further expand and enhance tie capability of each sensor component. The camera system shown 
has Ml 360 degree field of view capability which may be controlled manuaUy by remote control 
sig naU, may be programmed to pan me area on a time sequence, may tiack a moving nanspor. using 
OPS signals from the transport or by using image processing "ticking software" processing the 
camera image, or may be responsive to and activated by an event occurrence such as from sense 
distributed throughout the ramp areas, reporting activity over the LAN, in the well known manner. 
The range finder 66 permits the tracking system to locate objects in a precise manner and then 
pre v*control signals to permit accuse surveillance and monitoring of same, such as zooming the 
camera or positioning of otter sensor elements. Each sensor and/or camera may incorporate a 
motion sensor and/or an audio s*nsor activation device so that the system may be activated when a 

nnit that can detect the presence of an intruder and can be a device such as an infrared motion 
detector, a thermal sensor, an ulhasonic detector, a microwave detector, or any hybrid of two or more 
of these detectors "fused" together to gain better sensitivity and/or improved detection accuracy. A 
motion detector convertor may be incorporated to convert the signal from either a single motion 
detector sensor or a battery of sensors to digital form for processing and/or transmission to other 
system elements. Muitiple elements may be contained within a single sensor system package, or 
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transmitted over the LAN. The motion detector signal processor 

„. WeU-.no™ techniques may be indented to process the reducer data and detect 
^es over the se, thresholds that represent detection. The processor/compressor can aiso he 
configured to accept input from multiple sensors and process the inputs in a W manner. For 

oeforean.armconditionisbroadcast. ^ 

an even wouid trigger a specified, uni q ue aiarnr condition to he transmitted ha* to tie omer 

elements of the system. 

TypicaUy, the sensors wiU "sense" the presence of unauthorized activity and activate 

recordation of activity at the site. Motion detection may aiso he determined using video 

time/change techniques in the well-known manner. 

Fig. 18 is a perspective view of a multimedia alarm and response module for use in 
connection with me subject invention. The "box" 600 is shown in a wireless configuration and, 
tnerefore may be located in any desired location irrespective of the presence o, power and 
transmissioncabung. A self-contained power supply can be provided or, where available, the box 
may be connected to the local power source system with integral emergency hack-up in me weU 
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fcnown manner. The box .nay include any —.on of signal swipes such as "FIREW, 
..MEDICAL-604, -SECURITY^ and the l*e. When activated, a signa, win be .--o^ 

te user can communicate with One case station, a video camera 610 for provide* visua, 

be presented by tbe base station. An IR LED muminator 614 may be provided to permit ambient 
fflunumti o n. For certain *pes of appUcations, such as a fire, an audibie alarm 6X6 is incorporated 
and may be programmed to be automatical activate by pulling .he appropriate switch, or 

such as, by way of example, a smoke detector 61. may also be incorporated for automatically 

sending signals to the base station. 

Returning now to Fig. 1. each camera has a transmitter connected to the antenna 23 to 
receiver antenna 25 for transmitting an RF signa, o a wireiess LAN receiver 24 connected to the 
intranet 26 a, a location convenient for receiving the RF signal from the cameras 20, 22. The 
video and/or rmage signals captured and/or transmitted by the camera to the receiver are then 

example, a Pentium Cass PC, wherein fte raw data signa,s generated by tie cameras and/or 
transmitted by the receiver are processed for display. The monitor station 28 may also include 
an input device such as, by way of example, fte keyboard 32. An example o, a suitable wireless 
LAN is the Proxim "Range LAN2" system or me Interce, Prism system, consisting of PCM/CIA 
cards for small portable devices and a base station fta. provides an Ethernet connection to the 
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• ♦ ♦• r,r, "see and/or hear" data from the wireless camera(s) 
Intranet. The monitoring station can see ana/or ne 

mnint HPtprtinn such as by motion detection, 
continuously, periodically as programmed, or upon event detect™ such y 

audio detection, contact closure or other independent system .u*. U wil, be understood that any 
number of cameras, image sensors audio devices and other suitable sensors may be connected to 
the surveillance system using mis wireless system configuration. Of course, hardwired sensors 
can be incorporated in the system in common with the wireless sensors, without departing from 

the scope and spirit of the invention. 

m me typical application, the camera/sensor unit will include a combination of on board 
transducers or sensors as reaped to perform the full multimedia monitoring function, In the 
preferred embodiment, the camera includes a video camera for visual input based on conditions. 
For example, and as shown and described above, with reference to Fig. 17, me video camera 
my be monochrome or coior and may have an unage intensifier for operation in da* 
environments, and may indude infrared night vision capabuity. The lens may be a zoom lens or 
„«« iris and/or have tilt and/or pan capability. The video may be processed in mree distinct 
n^rs: (1) full motion stream-compressed to be delivered over the communications medium 
and typically the lowest resolution due to me volume of data required; (2) step video-higher 
resolution can be collected at a specified interva! for better identification of certain features or 
components in the range of me camera; (3, still frame images-highest resolution and faster than 
fnl, motion or step video. These three types of da. may be delivered m combination or may be 
any selected combination. For example, a low resolution continuous stream of highly compress* 
full motion data may be sent from me camera/sensor to a monitor station* drive a screen. This 
wU1 provide a typica, "closed circuit" monitor of the scene at the c^era/sensor. Authorized 
personnel may men command the sensor to capture and send a high resolution still image to the 
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• . .HHitinniii scrutiny If a sensor has been armed 
monitor station if something appears to reqmre additional scrutiny. 

„ event deletion and activation, the sensor may he activated to capture and send the event s ig nai 

„ f uses of this capability are within fte intended scope of me application. The camera/sensor also 

• ■ tn Qiinnort the transmission. For example, Internet 

includes adequate communications protocol to support tne u 

transmissions retire TCP/IP or PPP protocol Se.ected camera/sensors may be mobiie and it 
my be desirable to determine their .ocation hr the system. Any number of technics may be 
empioyed, for example, infrared beacon location systems, KF tria.gula.ion systems or GPS and 
the !*, The camera/sensor may be equipped with an external sensor input or a dry contact input 
t0 trigger the sensor to capture data and genera, an event. While the camera,sensor is Really 
equipped to accept externa, power such a low voltage DC, a baciorp internal power supply is 
desired. The wireless camera/sensor units include an RF subsystem, either externa, or internal, 
to communicate with the servers and monitors of the system. 

toman, cases the comprehensive camera/sensor wffl also include other devices such as an 
audio subsystem for collection of audio data, motion sensors, heat sensors, smolte and vapor 
detectors, and/or the like. Typically, fte camera sensor will include a temporary memory store. 
Tnis way, if signal transmission occurs upon the detection of the event, me camera/sensor can 
send da. renting to the time Just prior to the event, limited only by the on board memory 
capacity. A detailed description of a typical camera/sensor in accordance with the invention is 
provided in connection with Figs.9-13 herein. 

An enhancement of the system of Fig. 1 is shown in Fig. 2, wherein a router 34 is 
connect* to the Intranet as a gateway to the Merne, indicated by 36. This permits a remote 
monitor station 38 to be connected to the system anywhere where mere is Internet access. As wtft 
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♦ i« a CPU with an output or display 
♦ 09. the remote monitor station 38 is a tru wm 

the on-line momtor station 28, the remoic 

— — , and is P— we,, suited for — where s ig na, 

■ This svstem provides essentially worldwide and low 
over a large geographic region is required. This systemp 

wrtv of the system Any combination of local (Intranet) and remote 
cost access to the capability of the system. y 

(Internet) monitoring stations can be employed. 

■ via 3 for use in combination with a wireless LAN 
A server based system is shown m Fig. 3 for use m 

* nr Ad havine a transceiver and antenna 45 for 
system In this embodiment, a wireless momtor 44 having 

_icati — 1^ — — "* 
^hasaPDAorthelfce,^^ 

«* in the svstem and is connected to the Intranet in well known fashion. 
46 has also been incorporated in the system anu 

• r*, nf the wstem over the direct communication systems 
This substantially enhances the functionahty of the system 

♦ inration for data collection, alarm detection 
of Figs 1 and 2. The server provides a centralized location 

• ess control dispatch processing, logging functions and other specialized 
and processmg, access control, aisp^u v 

♦ ♦«rt.P«rver loesed and dispatched based on 
a ,.^ri herein Data is sent to the server, loggcu 

functions as described nerem. ^ 

• , w,™. nf the server is the ability to use standard Internet 
important, but not necessarily essential, feature of the server 

station to participate in the network if needed, without specialized software. 
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fig 4 is an expanded system using Interne, connection between the receiver 24 and the 
v arious other components such as the server 46 ^ one or multipie monitoring stattons 2 8-28n. 
Xneserverntayheiocatedvirtuanyanywhe.onthein— rne, This system would be a 

v. ™-™der suroorted a number of surveillance installations 
very useful configuration where a service provider suppone 

. The tonoloev of the network will be established 

for various client utilizing a single master server. The topology 

♦ of 9 « installation Multiple servers may also be used for size, 
by the geographic parameters of an installation, m v 

performance and redundancy issues. 

Where physical wiring is prohibited or diflicuK to impossMe „ ins* the wireiess carrier 
configuration of Fig. 5 is useM. In this configuration, any one or various coupon wireiess 
carriers, as Seated at 47, — the da. direct* * the Internet or Interne, gateway 48, where 
it „ distributed in the manner previous!, described. Any carrier with a gateway to tie Intranet 

• Hiatal cell ohones or an Internet connected satellite service 
connected two-way pager service, digital cell phones, or 

i u nther services that do not fundamentally have an 

such-as .Iridium service can also be used. Other services ma 

r«n he used once provided with an access path to the server or to an Internet 
Internet gateway , can be used once piuviut. 

g ateway. For example, conventional (non-digital) ceUular service does not have an Internet 
connection. However, an ISP can be called to provide the Internet connection. As shown inFrg. 
6 one or more wireiess monitors 44-44n can also he connected in mis fashion. For exampie, a 
PDA is outfitted with a CDPD in* and is adapted for receiving alarm events or multimedia data 
over a wide region wifrout wires. In mis configuration, the wireless carrier can aiso he a wire,ess 
LAN for smaller regions. A wireless lh* to a monitor device is particuiarly useful for mobne 
units such as a roving security officer. When an event occurs the roving officer can be notified 
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whil e on rounds with *e type o f even., the location and supplied wi* other multimedia da. such 

as images and/or maps and/or audio. 

Fig 7 is an elusive multimedia surveiUance system incorporating aU of the heretofore 

different geographic regions are monitored and connected to the system as indicated hy the three 
wlr e,ess systems Wireiess ! (W!>, Wireiess 2 (W2) and Wireiess 3 (W3). Each location has a 

. ^ v wi in wi 22- W2-20 W2-22; and W3-20, W3-22. Each 
plurality of cameras as indicated by Wl-20, Wl-22, W2 ai, w 

wireless carrier or wireless LAN Wl. W2, W3 — the signals to tie associated gateway Wi- 
48, W2^8, W3^8, respectively, via which the respective signals are introduced to the Internet 

nc M m P r« 50 and 52 may also be deployed. The 
36. As shown, hardwired sensors such as cameras 50 ana y 

~~r.a. routers ^4 34n to various monitor stations 
transmitted data is then routed through one or more routers 34, 34n 

28-28n via associated Intranets 26-26n. A cenrra. server 46 is connected to the Internet and 
operates as previously described. Additional servers may also be emp.oyed as previously 
describe. The Intranet faculties are protect by traditional firewalls and bridged through the 
routers. The focus of the system is the server that is shared by al, building surveillance sensors 

controUed aces, In a epical —on, the surveillance unit, in building 1 supported by 
Wireless 1 (Wl) both wired and wireless, will be mapped to a monitor station ta building 2 
support* by Wireless 2 (W2) and may also be mapped to a monitor station off site, and so on, 

useful, for examp.e, in monitoring school campuses from a remote tort*. Additiona, servers 
are often employed to provide redundancy. The specific networ* conation employed, 
including the type o, wireless carrier used (WLAN, CDPD, ceUular, pager and the hke, - 
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aetermined by geographic conation, cost and availability of the services to the particular 
geographic area and fte intended functions and scope of surveillance system. 

Fig. 8 illustrates examples of the server dispatching techniques which may be employed 
by the server supported multimedia surveiUance system of the subject invention. It is intended to 
ulustrate examples and is not intended to be all inclusive. As there shown, once the data is 
available to the server 46, it may be managed and processed in a number of different ways to 
.naxtaize the use of the dan, for monitoring, .ogging, archiving and response issues. The dam is 
typicauy sent to system monitors such as monitor 28 which may be on me local wireless Intranet 
or may be located elsewhere on the Internet. The data and various control signals may also be 
^patched via various communication systems such as public telephone lines 60 and Internet and 
BPS as indicated at 58. System information may men be distributed via normal means to pagers 
PLPn, voice cell phones CI and/or digital cell phones C2, to land line telephones Tl, to PDAs 
PD1, to voice mail service systems VI and to ofner remote monitoring stations Ml. Contiol 
signafe.response data and other data and toformauon may be introduced into the system via the 
same, systems. 

Tn a typica! operation, the server can be used to process various event notifications. For 
example, me server can arm sensors for detection of external events, audio detection, smoke or 
vapor detection, heat detection, ano/or video detection. If an activated sensor is triggered, the 
server can dispatch alarm conditions to several ways, including sending: 
messages to specifically assigned monitoring units; 

a telephone call to designated !and line numbers with an audio message describing 

the event; 
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a telephone call to a designated wireless .eiephone number wim an audio message 
describing the event; 

a numeric pager message with transmitted number signifying a voice mailbox 

m essage that contains a server generated audio message describing the event; 
a voice pager message with audio message describing the event; 
a ten pager message describing the event; 
a text message to a wireless PDA; 
E-mail message to a digital cellular phone; 

. ,„ . wirrieis PDA device, such as message showing a map, 
graphic message sent to a wireless run ucv»- , 

collected images and the like; 

E-mail to any e-mail accessed party via fixed terminals. 
^ server can also notify multiple parties, or can notify and wait for confirmation such 
as a tone back on a telephone or a reply on a two-way wireless system. If the confirmation does 

method for determining what stations a* units to notify is a predefined notification list mat maps 
sensors and types of events to particuiar parties. Primary parties and secondary or backup parties 
^identified. Time frames can be assigned. For example, perhaps three individuals are "on 
call" during one day and the notification is assigned based upon te time of day . A second method 
is to interface the time accounting system to see who is 'clocked in" at a particular time'. That 
individual would automatically be the priority contact during the period of time he is clocked in. 
GPS tracking of personnel can also be used to notify tire closes, personnel, tire Coses, momtor 
and ,o track and route me even, and are even, response. For example, personnel can be directed 
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• • ,™i manaeement to identify both the precise location of 
to the event using GPS signal monitoring and management 

the event and the closest response personnel. 

Tne serve, is the heart o, the system and not any centre, dispatch - — n of 

^^^^^^^^^""^^ 

all wired and wireless sensors is monitored and a r— on or Mure to respond is logged with 

• , -v. ■ at ARM" condition, for example) dispatched 
proper response triggering informanon (such as ALAKM 

t0 me appropriate monitoring stations and units. The monitor s.tions be monitored in a 
staji lar manner. Event detection such as sensor triggering or motion sensing is likewise 
monitored, logged and reacted to. The multimedia sensor da., such as audio, video, step video, 
and individual images are sent to the server by me camera/sensor unit by me avaiiahle 
M mmunica.ions circuit and archived for future use. The da. is time "stamped" with additional 
lotion such as time of collection, sensor identification, trigger condition if applicable, and 
ofcer significant da., for exampie environmental data such as temperature, humidity, Ugh, 

smoke, vapor and gas levels and the like. 

„ crosspoint switching the data, the system coliects input from a multitude of sensors and 

stations The server can perform as an N-by-N switch, mapping any combination of da. from any 
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sensor. Periodic images may be sent to a localized monitoring station. Motion detection from an 
armed sensor may facilitate sending text alarm conditions to a roving guard (such as can be done 
using an alpha-numeric pager, for example), maps, GPS direction and location, still frame and 
image display, as well as sending critical data and response information, and/or full alarm data to 
te localized monitoring station near the sensor and to a centralized company wide monitoring 
station for high leve. supervision. Other wo* stations in close proximity to the triggered sensor 
can be notified on the Intranet. An, matrix of mapping of the sensors, routing conditions and 
designated receiving monitor stations can be orchestrated by the server. 

The server can define zones for the sensors and monitor stations and manage and control 
oyzone. Sensor unit and monitor station firmware and software may be maintained at me server 



allowing for system wide maintenance and upgrade at a central location. 

The server can provide a sensor map and event map based on location of sensors and 
monitoring of events. Where moving sensors are included in the system, the server will keep 
tracic of the sensor either by tracking the location of the mobile sensor tough periodic polling, 
periodic sensor transmissions or GPS. Typically, the mobile sensor will be tracked on the map. 

The server will also provide other typical function such as area or zone monitoring to track 
personnel in the zone, controlling access authorization and controlling and monitoring logging and 
call back requests. The system can also generate a. a)arm if personnel and/or equipment enters 
an unauthorized area as monitored by the server and various devices. 

Figs. 9-13 illustrate various camera/sensor configurations and applications. A basic 
camera is shown in Fig. 9. This embodiment supports compressed fall motion video, step video, 
SU U frame video, compress* audio, geolocation, RF link, and optional sensor component input. 
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The processor of *e camera is jammed » opera, in the manner described to support the 
„ of the n—ia surveillance system of the present — . The camera 70 cohects 

compressed as JPEG, MPEG, and the like, as indicated at the video compressor 74. The 
compression and storage ftrnctions are control by the processor 78 as programme, and 
control* by f— 80. An audio sensor such as microphone 82 is also provided, and me 
ra w audio signal is compressed at audio compressor 84. Various additiona, sensors 86 may be 
^ as desired for each particular application. A storage memory 88 is provided for recailing 
. sensor da., as previous* described. Where desired, the system can also he programmed to 
record conUnuous.y to look at aU data after an even, has occurred. The output da. is transmitted 
via an output Une 90 and may be hard wired, or may be wheiess and connected via an RF link 92 
as radical. Mobile cameras win include a geolocation system 94, as previously described. 

Fig . 10 is a more detailed diagram of the camera electronics and shows the camera 70 as 
an analog capmre device communicating with an analog * digital (*>» convert 100 for 
generating a digitized signal to sent to the motion coder KB and the still frame compressor 104. 
As previously stated, the images are stored in the memory 106 for later recah and reconstruction 
of events. The data is combined a, combiner 108 and transmitted via communications interface 
U0 the communications channel or circuit 112 such as, by way of example, celMar, RF, land line 
telephone and the hke. The signal is then received at the communications interface 114 a, a 
remote location and stored for archiving purposes at mass storage unit 116. The data is extracted 
at „« and introduce* into a motion decoder 118 and still frame decompressor 120. The output 
is thenproduced on the monitor 122 through fhe D/A convertor 124. The switch 126 is used to 
seiec. either motion video via motion decoder 118 or stiH frame images via the still frame 
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compressor .20. The stU. frames are stored in memory 128 and may be hard copy printed a. 

printer 130. 

An alternative camera circuit configuration is shown in Fig. 11. In this embodiment, the 
digitized signa, from the A/D convenor !00 is unreduced into the MPEG encoder 132 and me 

™ 1 vi for <tnrine still images. The encoding and compression 
DSP134, which includes a memory 136 for stormg sun i^s« 

processes are control by a control processor 138. The signal are men combined a, a 
multiplexer 140 and mtroduced into a modem 142 for deUvery to the communications channels 
112. The remote modem 144 receives the transmitted signal and introduces it into a demultiplexer 
M6 from which it is introduced mto me MPEG decoder!50 and the DSP 152. which is controUed 
by the control processor 148. A stil! frame memory 160 is inciuded. The signal is me again 
multiplexed at 154 and converted to an anatog signal at D/A convenor 156 for display at monitor 
158. The still images may be hard copy printed at printer 162. 

As shown in Fig. 12, all of the functions of the camera system can be performed using me 
high performanc DSP 132 in combination with the camera 70 and the A./D converter 100 on the 
utpnt side and a memory 106. The output of me DSP is converted to an analog signal at D/A 
convertor 134. A digital camera may be substnuted for the anaiog camera described, eliminating 

the need for the D/A convertor 100. 

A remote triggering configuration is shown in Fig. 13. For example, a forHift truck 170 
may be outfitted with a geolocation system as indicated by the antenna 171. When the forHift is 
detected by the geographic locator to be in me field of view of me camera 70, the *rver will send 
a signal to the camera/sensor 70 via antenna 71 to activate the camera/sensor and coUect data for 
transmission to the system. Other activation sensors can be used as weil. For example, the 
proximity sensor 172 may activate the camera/sensor 70 whenever anything is within me range 
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or «- proximity -or. The pretty sensor signal canbe directly triggered » the camera or 
cm be sent bade to the server for management in accordance with programmed technic^. This 
wffl permit specific data collection of images in selected area and can provide a permanent archival 
record of any objects mat may have been in the vicinity of the camera during the time tie camera 

is activated. 

The invention provides a comprehensive surveillance scheme permitting both local and 
remote monitoring and management. While certain features and embodiments o, the invention 
have been described in detaii herein, it will be readily understood mat the invention encompasses 
an modification and enhancements with the scope and spirit of the Mowing claims. 
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CLAIMS 

! . A comprehensive multimedia surveillance system for monitoring an area or location 
without requiring hardwired sensors, the system comprising: 

a . a wireless sensor for detecting events in a area under surveillance and collecting 

data for recording the event; 

b . a wireless transmitter for sending the data; 

c. a wireless receiver for receiving the data at a location from the sensor; 

d. a monitor station; 

c ' a communication system for communicating the wireless receiver with the monitor 
station for transmitting 'the received data to the monitor station. 

2 . The surveillance system of claim 1, tether including a display screen associated 
with the monitor station for displaying the collected data. 

3. The surveillance sysu-n, of claim 1. further including a memory associated with the 
monitor station for archiving the collected data. 



4 . The surveillance system of claim 1, wherein the communication system for 
communicating the wireless receiver with the monitor station comprises a hardwired local area 

network. 
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5. The surveillance system of claim 1, wherein the communication system for 
communicating the wireless receiver with the monitor station comprises a wireless local area 



network. 



6. The surveillance system of claim 1, wherein the communication system for 
communication the wireless receiver with the monitor station comprises a gateway to the Internet 
and the Internet, and wherein the monitor station is at a remote location having Internet access. 

7. The surveillance system of claim 1, wherein the sensor includes an analog camera. 
> 8. The surveillance system of claim 7, wherein the sensor further includes an audio 



sensor.. 

9,. 

detector. 

10. 
detector. 

11. 



The surveillance system of claim 7, wherein the sensor further includes an event 



The surveiUance system of claim 9, wherein said event detector is a motion 



The surveillance system of claim 9, wherein said event detector is a heat sensor. 



12. The surveillance system of claim 9, wherein said event detector is a proximity 

switch. 
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13 The surveillance system 



of claim 9, wherein said event detector is a latch. 



14. The 



surveillance system of claim 1, wherein said sensor is a digital camera. 



15. A comprehensive multimedia surveillance system for monitoring an area 
without requiring hardwired sensors, the system comprising: 

a. a wireless camera sensor for visually monitoring an area under surveillance and 
collecting image data for recording the event; 

b. a wireless transmitter for sending the data; 

c. a wireless receiver for receiving the data at a location from the sensor; 

d . a monitor station including a display screen for displaying the event data; 

e . a communication system for communicating the wireless receiver with the monitor 
stationer transmitting the received data to the monitor station. 

16. The surveillance system of claim 15, the monitor station further including, a 
memory for storing the event data. 

17 . The surveillance system of claim 15, farther wherein the communication system is 
a local area network and the monitor station is on the local area network, and toe is farmer 
mduded an Internet gateway for transmitting the collected dam over the Internet and a second 
monitor station having Internet access and located on the Internet for receiving the coUected data. 
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18. The surveillance system of claim 15, further including a server for receiving and 
distributing the data, the server distributing the data to the monitor station via the local area 
network. 

19. A comprehensive multimedia surveillance system for monitoring an area or location 
without requiring hardwired sensors, the system comprising: 

a. a wireless sensor for detecting events in a area under surveillance and collecting 

data for recording the event; 

b. a wireless transmitter for sending the data; 

c. a wireless receiver for receiving the data at a location from the sensor; 

d. a server; 

e. a communication system for communicating the wireless receiver with the server 
for transmitting the received data to server. 

20. The surveillance system of claim 19, the server adapted, for processing and 
distributing the data, the surveillance system further including a monitor station in communication 
with the server for receiving the data. 

21. The surveillance system of claim 19, the monitor station further including a display 
screen for displaying data distributed to it by the server. 

22. The surveillance system of claim 19, the monitor station further including a 
memory for storing the data distributed to it by the server. 
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23. The surveillance system of claim 19, wherein the communication system is a 
hardwired local area network and the sever is on the local area network. 

24. The surveillance system of claim 19, wherein.the communication system is a 
wireless local area network and the server is on the local area network. 

25. The surveillance system of claim 19, wherein the communication system includes 
a gateway to the Internet and the server is on the Internet. 

26. The surveillance system of claim 25 , wherein the communication system further 
includes a local area network associated with the receiver and the monitor station is on the local 
area network. 

27. The surveillance system of claim 25, wherein the monitor station is on the Internet. 

28. The surveillance system of claim 1, the sensor further including a receiver for 
' receiving signals from the monitor station and the wireless receiver further including a transmitter 

for sending signals to the sensor, the monitor station further including an input device for sending 
control signals to the sensor. 
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29. The surveillance system of claim 28, further including a server, wherein the 
monitor station communicates only with the server and the wireless receiver communicates only 
with the server. 

30. The surveillance system of claim 19, the server adapted for collecting and 
distributing the collected data to distribution points, said distribution points comprising monitor 
stations for receiving and reacting to the collected data. 

■ 31. The surveillance system of claim 30, wherein a distribution point comprises a two- 
way communication device adapted for sending a response signal to the server when data is 
received. 

32. The surveillance system of claim 31, wherein said communication device is a voice 
cellphone. 

33 . The surveillance system of claim 3 1 , wherein said communication device is a land 
line telephone. 

34 . The surveillance system of claim 3 1 , wherein said communication device is a two- 
way pager. 

35. The surveillance system of claim 31, wherein the communication device is a voice 
mail service. 
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36. The surveillance system of claim 31, wherein the communication device is a digital 
cell phone 

37 . The surveillance system of claim 3 1 , wherein the communication device is a PDA. 

38. The surveillance system of claim 1 , further including wired sensors associated with 
the communication system. 

39. The surveillance system of claim 15, wherein the camera captures full motion 

video. 

40. The surveillance system of claim 39, wherein the camera is programmed to capture 
step action video. 

41 . The surveillance system of claim 39, wherein the camera is programmed to capture 
still frame images. 

42. The surveillance system of claim 31, wherein the server is programmed to send 
information messages to specific communication devices upon receipt of specified data from the 
sensor. 

43 . The surveillance system of claim 31 , wherein the communication device is mobile 
and further includes a geolocation device and wherein the server communicates to the mobile 
device in closest proximity to an event when event data is received from the sensor. 
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44. The surveillance system of claim 15, wherein the sensor is mobile and further 
includes a geolocation device for generating and sending tracking data to the server, and wherein 
the server tracks the location of the mobile sensor. 
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ABSTRACT 

A comprehensive, wireless multimedia surveillance and monitoring system is adapted for 
transmitting event data, video and/or image monitoring information, audio signals and other 
sensor and detector data over significant distances using digital data transmission over a wireless 
LAN, Intranet or Internet for automatic assessment and response including dispatch of response 
personnel. Both wired and wireless sensor systems may be employed. GPS dispatching is used 
to locate and alert personnel as well as to indicate the location of an event. Automatic mapping 
and dispatch permits rapid response. The wireless LAN connectivity permits local distribution 
of audio, video and image data over a relatively high bandwidth without requirement of a license 
and without relying on a common carrier and the fees associated therewith. The surveillance 
system may be interfaced with a WAN (wide area Intranet) or the Internet for providing a 
worldwide, low cost surveiUance system with virtually unlimited geographic application. 
Centralized monitoring stations have access to all of the surveillance data from various remote 
locations via the Internet or the WAN. A server provides a centralized location for data 
collection, alarm detection and processing, access control, dispatch processing, logging functions 
and other specialized functions. The server may be inserted virtually anywhere in the 
Intranet/Internet network. The topology of the network will be established by the geographic 
situation of the installation. Appropriate firewalls may be set up as desired. The server based 
system permits a security provider to have access to the sensor and surveillance data or to 
configure or reconfigure the system for any station on the Internet. 
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